Network optimization method and apparatus, image processing method and apparatus, and storage medium

ABSTRACT

The present disclosure relates to a network optimization method and apparatus, an image processing method and apparatus, and a storage medium. The network optimization method includes: obtaining an image sample group; obtaining a first feature and a second feature of an image in the image sample group, and obtaining a first classification result by using the first feature of the image; performing feature exchange processing on an image pair in the image sample group to obtain a new image pair; obtaining a first loss value of the first classification result, a second loss value of the new image pair, and a third loss value of first features and second features of the new image pair in a preset manner; and adjusting parameters of a neural network at least according to the first loss value, the second loss value, and the third loss value until a preset requirement is met.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure is a bypass continuation of and claims priorityunder 35 U.S.C. § 111(a) to PCT Application. No. PCT/CN2019/118922,filed on Nov. 15, 2019, which claims priority to Chinese PatentApplication No. 201910036096.X, filed with the Chinese Patent Office onJan. 30, 2019 and entitled “NETWORK OPTIMIZATION METHOD AND APPARATUS,IMAGE PROCESSING METHOD AND APPARATUS, AND STORAGE MEDIUM”, each ofwhich is incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to the field of network optimization, andin particular, to a network optimization method and apparatus, an imageprocessing method and apparatus, and a storage medium.

BACKGROUND

Pedestrian re-identification aims to learn discriminative features forpedestrian retrieval and matching. Generally, factors such as pedestrianpose diversity and background diversity in an image data set affect theextraction of identity features. Currently, in related technologies,decomposition features are extracted by using a deep neural network foridentity recognition.

SUMMARY

Embodiments of the present disclosure provide technical solutions fornetwork optimization.

According to one aspect of the present disclosure, a networkoptimization method is provided; the method is used for optimizing aneural network, and includes:

obtaining an image sample group, where the image sample group includesan image pair formed by images of a same object and an image pair formedby images of different objects; obtaining a first feature and a secondfeature of an image in the image sample group, and obtaining a firstclassification result by using the first feature of the image, where thefirst feature includes an identity feature, and the second featureincludes an attribute feature; performing feature exchange processing onan image pair in the image sample group to obtain a new image pair,where the feature exchange processing is to generate a new first imageby using a first feature of a first image and a second feature of asecond image in the image pair, and to generate a new second image byusing a second feature of the first image and a first feature of thesecond image; obtaining a first loss value of the first classificationresult, a second loss value of the new image pair, and a third lossvalue of first features and second features of the new image pair in apreset manner; and adjusting parameters of a neural network at leastaccording to the first loss value, the second loss value, and the thirdloss value until a preset requirement is met.

According to a second aspect of the present disclosure, an imageprocessing method is provided, including:

receiving an input image; recognizing a first feature of the input imageby means of a neural network model; and determining an identity of anobject in the input image based on the first feature; where the neuralnetwork model is a network model obtained after optimization processingthrough the network optimization method according to any item in thefirst aspect.

According to a third aspect of the present disclosure, an imageprocessing apparatus is provided, including:

an obtaining module, configured to obtain an image sample group, wherethe image sample group includes an image pair formed by images of a sameobject and an image pair formed by images of different objects; afeature coding network module, configured to obtain a first feature anda second feature of an image in the image sample group; a classificationmodule, configured to obtain a first classification result according tothe first feature of the image; a generation network module, configuredto perform feature exchange processing on an image pair in the imagesample group to obtain a new image pair, where the feature exchangeprocessing is to generate a new first image by using a first feature ofa first image and a second feature of a second image in the image pair,and to generate a new second image by using a second feature of thefirst image and a first feature of the second image; a loss valueobtaining module, configured to obtain a first loss value of the firstclassification result, a second loss value of the new image pair, and athird loss value of first features and second features of the new imagepair in a preset manner; and an adjustment module, configured to adjustparameters of the neural network at least according to the first lossvalue, the second loss value, and the third loss value until a presetrequirement is met.

According to a fourth aspect of the present disclosure, an imageprocessing apparatus is provided, and includes:

a receiving module, configured to receive an input image; a recognitionmodule, configured to recognize a first feature of the input image bymeans of a neural network model; and an identity determination module,configured to determine an identity of an object in the input imagebased on the first feature; where the neural network model is a networkmodel obtained after optimization processing through the networkoptimization method according to any item in the first aspect.

According to a fifth aspect of the present disclosure, an electronicdevice is provided, including:

a processor; and a memory configured to store processor-executableinstructions; where the processor is configured to perform the methodaccording to any item in the first aspect and the second aspect.

According to a sixth aspect of the present disclosure, acomputer-readable storage medium is provided, and has computer programinstructions stored thereon, where when the computer programinstructions are executed by a processor, the method according to anyitem in the first aspect and the second aspect is implemented.

According to one aspect of the present disclosure, a computer program isprovided, where the computer program includes a computer-readable code,and when the computer-readable code runs in an electronic device, aprocessor in the electronic device performs the above-mentioned networkoptimization method.

It should be understood that the foregoing general descriptions and thefollowing detailed descriptions are merely exemplary and explanatory,but are not intended to limit the present disclosure.

The other features and aspects of the present disclosure can bedescribed more clearly according to the detailed descriptions of theexemplary embodiments in the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings here incorporated in the present specificationand constituting a part of the present specification illustrate theembodiments consistent with the present disclosure, and are intended toexplain the technical solutions of the present disclosure together withthe specification.

FIG. 1 is a flowchart of a network optimization method according toembodiments of the present disclosure;

FIG. 2 is a flowchart of step S200 in a network optimization methodaccording to embodiments of the present disclosure;

FIG. 3 is a flowchart of step S300 in a network optimization methodaccording to embodiments of the present disclosure;

FIG. 4 is a flowchart of step S303 in a network optimization methodaccording to embodiments of the present disclosure;

FIG. 5 is a flowchart of step S400 in an image processing methodaccording to embodiments of the present disclosure;

FIG. 6 is a schematic diagram of a process of performing networkoptimization processing by using a first type of sample according toembodiments of the present disclosure;

FIG. 7 is a schematic diagram of a process of performing networkoptimization processing by using a second type of sample according toembodiments of the present disclosure;

FIG. 8 is a flowchart of an imaging processing method according toembodiments of the present disclosure;

FIG. 9 is a block diagram of a network optimization apparatus accordingto embodiments of the present disclosure;

FIG. 10 is a block diagram of an image processing apparatus according toembodiments of the present disclosure;

FIG. 11 is a block diagram of an electronic device 800 according toembodiments of the present disclosure; and

FIG. 12 is a block diagram of an electronic device 1900 according toembodiments of the present disclosure.

DETAILED DESCRIPTION

The various exemplary embodiments, features, and aspects of the presentdisclosure are described below in detail with reference to theaccompanying drawings. The same reference numerals in the accompanyingdrawings represent elements having the same or similar functions.Although the various aspects of the embodiments are illustrated in theaccompanying drawings, unless stated particularly, it is not required todraw the accompanying drawings in proportion.

The special word “exemplary” here means “used as examples, embodiments,or descriptions”. Any “exemplary” embodiment given here is notnecessarily construed as being superior to or better than otherembodiments.

The term “and/or” herein describes only an association relationshipdescribing associated objects and represents that three relationshipsmay exist. For example, A and/or B may represent the following threecases: only A exists, both A and B exist, and only B exists. Inaddition, the term “at least one” herein indicates any one of multiplelisted items or any combination of at least two of multiple listeditems. For example, including at least one of A, B, or C may indicateincluding any one or more elements selected from a set consisting of A,B, and C.

In addition, numerous details are given in the following detaileddescription for the purpose of better explaining the present disclosure.A person skilled in the art should understand that the presentdisclosure may also be implemented without some specific details. Insome examples, methods, means, elements, and circuits well known to aperson skilled in the art are not described in detail so as to highlightthe subject matter of the present disclosure.

The embodiments of the present disclosure provide a network optimizationmethod, which is used for training a neural network or other machinelearning networks. For example, the network optimization method may beused in a process of training a machine learning network in scenarios inwhich face recognition, identity verification, and so on are performedon a target user, and may also be used in a process of training anetwork with a relatively high precision requirement such as identifyingauthenticity of images. The present disclosure does not limit specificapplication scenarios, and all the processes implemented using thenetwork optimization method provided in the present disclosure arewithin the scope of protection of the present disclosure. In theembodiments of the present disclosure, a neural network is taken as anexample for description, but no specific limitation is made thereto.After the network optimization method in the embodiments of the presentdisclosure is used for training, recognition precision of a characterobject in a network can be improved, and no auxiliary information otherthan an input image is required. Therefore, the network optimizationmethod is simple and convenient.

The network optimization solution provided in the embodiments of thepresent disclosure is executed by a terminal device, a server, or othertypes of electronic devices, where the terminal device may be UserEquipment (UE), a mobile device, a user terminal, a terminal, a cellularphone, a cordless phone, a Personal Digital Assistant (PDA), a handhelddevice, a computing device, a vehicle-mounted device, a wearable device,and the like. In some possible implementations, the network optimizationmethod is implemented by a processor by invoking computer-readableinstructions stored in a memory.

FIG. 1 is a flowchart of a network optimization method according toembodiments of the present disclosure. As shown in FIG. 1, the networkoptimization method in the embodiments of the present disclosureincludes:

S100: obtaining an image sample group, where the image sample groupincludes an image pair formed by images of a same object and an imagepair formed by images of different objects;

S200: obtaining a first feature and a second feature of an image in theimage sample group, and obtaining a first classification result by usingthe first feature of the image, where the first feature includes anidentity feature, and the second feature includes an attribute feature;

S300: performing feature exchange processing on an image pair in theimage sample group to obtain a new image pair, where the featureexchange processing is to generate a new first image by using a firstfeature of a first image and a second feature of a second image in theimage pair, and to generate a new second image by using a second featureof the first image and a first feature of the second image;

S400: obtaining a first loss value of the first classification result, asecond loss value of the new image pair, and a third loss value of firstfeatures and second features of the new image pair in a preset manner;and

S500: adjusting parameters of a neural network at least according to thefirst loss value, the second loss value, and the third loss value untila preset requirement is met.

In the embodiments of the present disclosure, if a neural network istrained through the embodiments of the present disclosure, an imagesample group is first input to the neural network, and the image samplegroup is taken as a sample image for training the neural network. In theembodiments of the present disclosure, the image sample group includestwo types of image samples. The first type of samples are image pairsformed by different images of a same object, and the second type ofsamples are image pairs formed by different images of different objects.That is, in the first type of samples, images in each image pair aredifferent images of the same object, and in the second type of samples,images in each image pair are different images of different objects.Each image pair includes two images, such as the following first imageand second image. In addition, in the embodiments of the presentdisclosure, the neural network is trained by using the two types ofimage samples.

Further, at least one image in the image sample group in the embodimentsof the present disclosure has a corresponding identity, and the identitycorresponds to an object in the image, and is used for recognizing anidentity of a character object in the image. In the embodiments of thepresent disclosure, at least one image in the image sample group has areal classification label corresponding to an object corresponding tosaid image, and the real classification label is represented by amatrix. The accuracies of classification results of a neural networkmodel are compared according to the real classification label, forexample, a corresponding loss value is determined.

In some possible implementations, a manner of obtaining the image samplegroup includes: receiving, by using a communications component, an imagesample group transmitted by another electronic device, for example,receiving an image sample group from a server, a mobile phone, or anycomputer device. At least one image in the image sample group ismultiple image pairs obtained after performing coding processing on avideo image collected by a camera, and is not limited specificlimitation in the present disclosure.

After the image sample group is obtained, a specific optimizationprocess of the neural network is performed. At step S200, first featuresand second features of the first image and the second image in eachimage pair are first recognized. The first feature includes an identityfeature of the object in the image, such as a color, a shape, and anornament feature of a dress. The second feature is a feature other thanthe first feature, for example, an attribute feature, and includes apose feature, a background feature, and an environment feature of thecharacter object. A manner of obtaining the first feature and the secondfeature is exemplified below.

FIG. 2 is a flowchart of step S200 in a network optimization methodaccording to embodiments of the present disclosure. Obtaining the firstfeature and the second feature of the image in the image sample group,and obtaining the first classification result by using the first featureof the image includes:

S201: inputting two images in the image pair to an identity codingnetwork module and an attribute coding network module of the neuralnetwork;

S202: obtaining first features of the two images in the image pair byusing the identity coding network module, and obtaining second featuresof the two images in the image pair by using the attribute codingnetwork module; and

S203: obtaining a first classification result corresponding to the firstfeature by using a classification module of the neural network.

The neural network in the embodiments of the present disclosure includesthe identity coding network module and the attribute coding networkmodule. The identity coding network module is configured to recognize anidentity feature of an object in an image, and the attribute codingnetwork module is configured to recognize an attribute feature of theobject in the image. Therefore, at least one image pair in the obtainedimage sample group is respectively input into the identity codingnetwork module and the attribute coding network module. The identitycoding network module obtains the first features of the two images inthe received image pair by means of the identity coding network module,and obtains the second features of the two images in the received imagepair by means of the attribute coding network module. For example, ifthe two images in the input image pair are respectively represented by Aand B, the first feature of A obtained by means of the identity codingnetwork module is Au, the first feature of B obtained by means of theidentity coding network module is Bu, the second feature of A obtainedby means of the attribute coding network module is Av, and the secondfeature of B obtained by means of the attribute coding network module isBy.

The identity coding network module may extract the first feature in theimage by using a preset character feature extraction algorithm, orinclude module units such as a convolution module and a pooling module,to perform the obtaining of the first feature. The structure of theidentity coding network module is not specifically limited in theembodiments of the present disclosure, and can be used as the identitycoding network module in the embodiments of the present disclosure aslong as it can extract the first feature in the image.

Similarly, the attribute coding network module may also extract thesecond feature in the image by using a preset pose and backgroundfeature algorithm, or include module units such as a convolution module.The structure of the attribute coding network module is not specificallylimited in the embodiments of the present disclosure, and can be used asthe attribute coding network module in the embodiments of the presentdisclosure as long as it can extract the second feature in the image.

After the first features and the second features of the two images inthe image pair are extracted, in the embodiments of the presentdisclosure, an operation of performing classification recognition byusing the first features is performed, and subsequent feature exchangeprocessing is also performed.

The neural network in the embodiments of the present disclosure furtherincludes a classification module. An output side of the identity codingnetwork module is connected to an input side of the classificationmodule, to receive the first feature output by the identity codingnetwork module. The classification module obtains a first classificationresult according to the received first feature. The first classificationresult is used for indicating a prediction result of an identitycorresponding to the first feature, and the prediction result ispresented in the form of a matrix. An element of the matrix is aprobability of predicting an object identity. The composition of theclassification module in the embodiments of the present disclosure isvoluntarily set, and the classification module can obtain the firstclassification result corresponding to the first feature by using a setclassification principle, and can be used as the embodiments of thepresent disclosure as long as it can perform the classification of thefirst feature. After the first classification result is obtained, afirst loss value corresponding to the first classification result isobtained, a loss value of the neural network is further determinedaccording to the first loss value, and feedback adjustment are performedon parameters in the network.

In addition, after the first feature and the second feature of eachimage are obtained, feature exchange processing between every two imagesin the image pair is performed. As described in the foregoingembodiments, the feature exchange processing is exchanging the secondfeature of the first image and the second feature of the second image inthe image pair, and obtaining a new image based on the first feature andthe exchanged second feature.

Through feature exchange processing, the first feature of one image iscombined with the second feature of the other image to form a new image,and classification is performed by using the new image, to effectivelyrecognize a character identity based on the identity feature, therebyreducing the impact of attributes such as the background and the pose.

FIG. 3 is a flowchart of step S300 in an image processing methodaccording to embodiments of the present disclosure. Performing featureexchange processing on the image pair in the image sample group toobtain the new image pair includes:

S301: inputting the image pair of the image sample group to a generationnetwork module of the neural network; and

S302: performing the feature exchange processing on the image pair inthe image sample group by means of the generation network module, toobtain the new image pair.

The neural network in the embodiments of the present disclosure furtherincludes the generation network module. The generation network moduleperforms feature exchange processing on the first feature and the secondfeature that are obtained by the identity coding network module and theattribute coding network module, and obtains the new image according tothe exchanged features. Specifically, as described in the foregoingembodiments, the image sample group input in the embodiments of thepresent disclosure includes two types of image sample groups. An imagepair in a first type of sample is images of a same object. For the imagepair in the first type of sample, the feature exchange processing isperformed on the images in each image pair once in the embodiments ofthe present disclosure.

For the first type of sample, performing feature exchange processing onthe images in the image sample group to obtain the new image pairincludes: performing feature exchange processing on the images in theimage pair once, to obtain the new image pair. This process includes:

generating a new first image by using the first feature of the firstimage and the second feature of the second image in the image pair, andgenerating a new second image by using the second feature of the firstimage and the first feature of the second image.

Because two images in the image pair in the first type of sample aredifferent images of the same object, the new image obtained after thefeature exchange processing is still an image of the same object. Afterthe feature exchange processing is completed, the loss value of theneural network is determined by using a difference between the obtainednew image and a corresponding original image as well as a differencebetween a first feature and a second feature of the new image and afirst feature and a second feature of the corresponding original image,and recognition and classification are directly performed according tothe generated new image. In this case, the generated new image pair isinput to the classification module, and classification is performed toobtain a second classification result.

For example, the image pair in the first type of sample includes animage A and an image B, the first feature of A obtained by means of theidentity coding network module is Au, the first feature of B obtained bymeans of the identity coding network module is Bu, the second feature ofA obtained by means of the attribute coding network module is Av, andthe second feature of B obtained by means of the attribute codingnetwork module is By. A and B are respectively the first image and thesecond image of the same object, and the first image and the secondimage are different. During the feature exchange processing, a new firstimage A′ is obtained by using the first feature Au of A and the secondfeature By of B, and a new second image B′ is obtained by using thefirst feature Bu of B and the second feature Av of A.

As described above, the neural network in the embodiments of the presentdisclosure includes the generation network module, and the generationnetwork module is configured to generate a new image based on thereceived first feature and the received second feature. For example, thegeneration network module includes at least one convolution unit, orincludes other processing units, and an image corresponding to the firstfeature and the second feature is obtained by means of the generationnetwork module. That is, a process of exchanging the second features andgenerating an image based on the exchanged features is completed by ageneration network.

Through the foregoing feature exchange processing, a new image is formedby exchanging second features of two images, so that identity-relatedfeatures and identity-independent features can be successfullyseparated. By training the neural network in this manner, recognitionprecision of the neural network for identity features can be improved.

In addition, the image sample group in the embodiments of the presentdisclosure further includes a second type of sample group, and an imagepair in the second type of sample group includes images of differentobjects. For the image pair in the second type of sample, the featureexchange processing is performed on the images in each image pair twicein the embodiments of the present disclosure.

For the second type of sample group, FIG. 4 is a flowchart of step S303of a network optimization method according to embodiments of the presentdisclosure. If the input image pair includes images of differentobjects, performing feature exchange processing on the image in theimage sample group to obtain the new image pair includes: performingfeature exchange processing on the image in the image pair twice, toobtain the new image pair. This process includes:

S3031: generating a first intermediate image by using the first featureof the first image and the second feature of the second image in theimage pair, and generating a second intermediate image by using thesecond feature of the first image and the first feature of the secondimage; and

S3032: generating a new first image by using a first feature of thefirst intermediate image and a second feature of the second intermediateimage, and generating a new second image by using a second feature ofthe first intermediate image and a first feature of the secondintermediate image.

For example, the first feature of A obtained by means of the identitycoding network module is Au, the first feature of B obtained by means ofthe identity coding network module is Bu, the second feature of Aobtained by means of the attribute coding network module is Av, and thesecond feature of B obtained by means of the attribute coding networkmodule is By. A and B are respectively the first image and the secondimage of different objects. When the feature exchange processing isperformed for the first time, a new first intermediate image A′ isobtained by using the first feature Au of A and the second feature By ofB, and a new second intermediate image B′ is obtained by using the firstfeature Bu of B and the second feature Av of A. Correspondingly, whenthe feature exchange processing is performed for the second time, afirst feature A_(u)′ and a second feature A_(v)′ of the firstintermediate image A′ as well as a first feature B_(u)′ and a secondfeature B_(v)′ of the second intermediate image B′ are respectivelyobtained by using the identity coding network module and the attributecoding network module again. Exchange processing of the second featureA_(v)′ of the first intermediate image A′ and the second feature B_(v)′of the second intermediate image B′ is further performed by using thegeneration network, a new first image A″ is generated by using the firstfeature A_(u)′ of the first intermediate image A′ and the second featureB_(v)′ of the second intermediate image B′, and a new second image B″ isgenerated by using the second feature A_(v)′ of the first intermediateimage A′ and the first feature B_(u)′ of the second intermediate imageB′.

Through the above feature exchange processing performed twice, a newimage is formed by exchanging second features of two images. Thedifference from the process of training the image pair of the sameidentity object lies in that: for the second type of sample, becausethere is no direct pixel-level supervision after the first featureexchange processing, the second feature exchange processing isperformed, and an image corresponding to the original image isgenerated. This process may be a cyclic generation process.

After the feature exchange processing is completed, a difference betweenthe obtained new image and a corresponding original image, and adifference between a first feature and a second feature of the new imageand a first feature and a second feature of the corresponding originalimage are used. In addition, the first feature of the new image is alsoinput to the classification module for classification processing toobtain the second classification result. For a case of the first type ofsample, a second classification result of the first feature of the finalnew image is directly obtained. For a case of the second type of sample,in addition to the second classification result of the first feature ofthe final new image, a second classification result of the first featureof the intermediate image is also obtained. In the embodiments of thepresent disclosure, the neural network is optimized according to thesecond classification result, a difference between the new image and theoriginal image, and a difference between the features. That is, in theembodiments of the present disclosure, feedback adjustment is performedon the neural network according to a loss value of an output resultobtained by each network module of the neural network, until the lossvalue of the neural network meets a preset requirement. If the lossvalue is less than a loss threshold, it is determined that the presetrequirement is met. A loss function of the neural network in theembodiments of the present disclosure is related to loss functions ofnetwork modules, for example, is a weighted sum of the loss functions ofmultiple network modules. Based on this, the loss value of the neuralnetwork is obtained by using the loss value of each network module,whereby adjusting parameters of each network module of the neuralnetwork until the preset requirement that the loss value is less thanthe loss threshold is met, and the loss threshold is set according tothe requirements of a person skilled in the art. No specific limitationis made thereto in the present disclosure.

Hereinafter, the feedback adjustment process in the embodiments of thepresent disclosure is described in detail.

After the first feature of the image is obtained by means of theidentity coding network module, the classification module obtains thefirst classification result according to the first feature. In theembodiments of the present disclosure, a first loss value of the firstclassification result obtained by using the first feature obtained bymeans of the identity coding network module is obtained in a firstpreset manner. FIG. 5 is a flowchart of step S400 in an image processingmethod according to embodiments of the present disclosure. The processof obtaining the first loss value includes:

S401: obtaining the first classification result of the first feature bymeans of the identity coding network module; and

S402: obtaining the first loss value in a first preset manner accordingto the first classification result and a real classification resultcorresponding to the image in the image sample group.

As described in the foregoing embodiments, at step S200, when the firstfeature of the image in the sample is obtained, classificationrecognition of the first feature is performed by means of theclassification module, to obtain the first classification resultcorresponding to the first feature. The first classification result isrepresented in the form of a matrix, and each element in the matrix isrepresented as a probability corresponding to each identity label. Afirst difference is obtained by comparing the first classificationresult with a real classification result, and the first difference isdetermined as the first loss value in the embodiments of the presentdisclosure. Alternatively, in other embodiments, the firstclassification result and the real classification result are input intoa first loss function of the classification module to obtain the firstloss value. No specific limitation is made thereto in the presentdisclosure.

In the embodiments of the present disclosure, when training the neuralnetwork by means the first type of sample and by means of the secondtype of sample, the loss functions used may be the same or different. Inaddition, in the embodiments of the present disclosure, summationprocessing is performed on the loss value of the neural network obtainedby training through the first type of sample and the loss value of theneural network obtained by training through the second type of sample,to obtain a final loss value of the neural network, and feedbackadjustment processing is performed on the network by using the lossvalue, where in the feedback adjustment process, the parameters of eachnetwork module of the neural network are adjusted, or only theparameters of some of the network modules are adjusted. No specificlimitation is made thereto in the present disclosure.

First, in the embodiments of the present disclosure, the first lossvalue of the first classification result obtained by using the firstfeature obtained by means of the identity coding network module isobtained in the first preset manner. The expression of the first presetmanner is as presented by formula (1).L _(c)=−Σ_(i=1) ^(N) L⊙ log(C(I))  Formula (1)

where ⊙ represents element multiplication, C(I) represents anN-dimensional prediction feature vector corresponding to the firstclassification result, L is an N-dimensional feature vectorcorresponding to a real label of a corresponding original image (a realclassification result), L_(c) is a first loss value corresponding to thefirst loss function, and i is a variable greater than or equal to 1 andless than or equal to N.

The first loss value of the first classification result obtained by theclassification module may be obtained in the foregoing manner. In theembodiments of the present disclosure, feedback adjustment is performedon parameters of the identity coding network module, the attributecoding network module, and the classification module according to thefirst loss value, or an overall loss value of the neural network isdetermined according to the first loss value and the loss values ofother network modules, and unified feedback adjustment is performed onat least one network module of the neural network. No limitation is madethereto in the present disclosure.

Secondly, in the embodiments of the present disclosure, processing isalso performed on the new image pair generated by the generation networkmodule, to obtain a second loss value of the new image pair and a thirdloss value of a corresponding feature. The manner of determining thesecond loss value includes: obtaining, in the second preset manner, asecond loss value of the new image pair obtained by means of the networkgeneration module relative to an original image pair.

In the embodiments of the present disclosure, the new image pair isobtained by means of the generation network. In the embodiments of thepresent disclosure, the second loss value is determined according to adifference between the new image pair and the original image pair.

For the first type of sample, the expression of the second preset manneris as represented by formula (2):L _(ir) =−∥{circumflex over (X)} _(v) −X _(v)∥₁ +∥{circumflex over (X)}_(u) −X _(u)∥₁  Formula (2)

where X_(u) and X_(v) are respectively a first image and a second imagein the original image pair, {circumflex over (X)}_(u) and {circumflexover (X)}_(v) are respectively a new first image and a new second imagein the new image pair, L_(ir) is a second loss value corresponding to asecond loss function, and ∥ ∥1 denotes a norm 1.

The second loss value corresponding to the new image pair generated bythe generation network module for the first type of sample is obtainedin the foregoing manner.

For the second type of sample, the expression of the second presetmanner is as represented by formula (3):L _(cr) =∥{tilde over (X)} _(u) −X _(u)∥₁ +∥{tilde over (Y)} _(w) −X_(w)∥₁  Formula (3)

where X_(u) is the first image in the original image pair, {tilde over(X)}_(u) is the first image in the new image pair, Yw is the secondimage in the original image pair, and {tilde over (Y)}_(w) is the secondimage in the new image pair.

In addition, in the embodiments of the present disclosure, a third lossvalue corresponding to a feature of the new second image pair is furtherobtained, and the third loss value is obtained in a third preset manner.

The expression of the third preset manner is as represented by formula(4):

$\begin{matrix}{L_{s} = {{- \frac{I_{Xu}^{T}I_{\hat{X}u}}{{I_{Xu}}_{2}{I_{\hat{X}u}}_{2}}} - \frac{I_{Xv}^{T}I_{\hat{X}v}}{{I_{Xv}}_{2}{I_{\hat{X}v}}_{2}}}} & {{Formula}\mspace{14mu}(4)}\end{matrix}$

where I_(Xu) represents a first feature of the first image X_(u) in theoriginal image pair, I_({circumflex over (X)}u) represents a firstfeature of the new first image {circumflex over (X)}_(u), I_(Xv)represents a second feature of the second image X_(v) in the originalimage pair, I_({circumflex over (X)}v) is a second feature of the newsecond image {circumflex over (X)}_(v), T is a transposition operation,L_(s) is a loss value corresponding to the third loss function, and ∥ ∥2represents a norm 2.

In the foregoing manner, the classification module obtains the thirdloss value corresponding to the feature of the new image pair generatedby the generation network module.

Similarly, in the embodiments of the present disclosure, feedbackadjustment may be performed on the parameters of the network generationmodule respectively according to the second loss value and the thirdloss value, and feedback adjustment may also be performed on themultiple network modules of the neural network simultaneously. Forexample, in some possible implementations of the present disclosure, theloss value of the neural network is obtained by using a weighted sum ofthe first loss value, the second loss value, and the third loss value.That is, the loss function of the neural network is the weighted sum ofthe first loss function, the second loss function, and the third lossfunction. The weight of each loss function is not specifically limitedin the present disclosure, and can be set by a person skilled in the artaccording to requirements. If the obtained loss value is greater thanthe loss threshold, feedback adjustment is performed on the parametersof the multiple network modules. The training is terminated when theloss value is less than the loss threshold. In this case, neural networkoptimization is completed. In addition, in the embodiments of thepresent disclosure, the first loss function, the second loss function,and the third loss function during training based on the image pair ofthe first type of sample may be different from the first loss function,the second loss function, and the third loss function during trainingbased on the image pair of the second type of sample-, but this is not aspecific limitation in the present disclosure.

In addition, to enhance the training precision of the neural network inthe embodiments of the present disclosure, the neural network in theembodiments of the present disclosure further includes a discriminationnetwork module. The discrimination network module is configured todetermine reality (a label feature of reality) of the generated newimage pair, and a fourth loss value corresponding to the generated newimage pair determined by the discrimination network module is obtainedaccording to the reality. A discrimination network and the generationnetwork in the embodiments of the present disclosure constitute agenerative adversarial network. For the specific structure of thegenerative adversarial network, a person skilled in the art could selecta proper structure for configuration according to the existing technicalmeans. No specific limitation is made thereto in the present disclosure.In the embodiments of the present disclosure, the generated new imagepair is input to the discrimination network module of the neuralnetwork, and the fourth loss value of the new image pair is obtained ina fourth preset manner.

The expression of the fourth preset manner is as represented by formula(5):L _(adv)=−(E[log D(X)]+E[log(1−D({circumflex over (X)}))])  Formula (5)

where D represents a model function of the discrimination networkmodule, E[ ] represents an expectation, X represents an original imagecorresponding to a new image, i.e., a real image, {circumflex over (X)}represents the new image input to the discrimination network module,D(X) represents a label feature of the discrimination network module forthe real image, and D({circumflex over (X)}) represents a label featureof the discrimination network module for the input new image. An elementin D({circumflex over (X)}) is a value from 0 to 1. The closer the valueis to 1, the higher the reality of the element is.

In the embodiments of the present disclosure, the training process ofthe discrimination network module may be performed separately, that is,the generated new image and the corresponding real image are input tothe discrimination network module, and the discrimination network moduleis trained based on the fourth loss function until the loss valuecorresponding to the fourth loss function is lower than a loss thresholdrequired for training.

In other possible embodiments, the discrimination network module mayalso be trained simultaneously with the identity coding network module,the attribute coding network module, and the generation network module.Accordingly, in step S400 of the embodiments of the present disclosure,the loss value of the neural network may also be obtained by using thefirst loss value, the second loss value, the third loss value, and thefourth loss value. That is, the loss function of the neural network is aweighted sum of the first loss function, the second loss function, thethird loss function, and the fourth loss function; the weight of eachloss function is not specifically limited in the present disclosure, andcan be set by a person skilled in the art according to requirements. Ifthe obtained loss value is greater than the loss threshold, feedbackadjustment is performed on the parameters of multiple network modules ofthe neural network. The training is terminated when the loss value isless than the loss threshold. In this case, neural network optimizationis completed.

In addition, in the embodiments of the present disclosure, the firstloss function, the second loss function, and the third loss functionduring training based on the image pair of the first type of sample maybe different from the first loss function, the second loss function, andthe third loss function during training based on the image pair of thesecond type of sample, but this is not a specific limitation in thepresent disclosure.

In some possible embodiments of the present disclosure, if the imagesample group input to the identity coding network module and theattribute coding network module is an image pair of a same object (thefirst type of sample), a first network loss value of the neural networkis obtained in a fifth preset manner based on the first loss value, thesecond loss value, the third loss value, and the fourth loss value. Theexpression of the fifth preset manner is as represented by formula (6):L _(int ra) =L _(c)+λ_(ir) L _(ir)+λ_(s) L _(s)+λ_(adv) L_(adv)  Formula (6)

where λ_(ir), λ_(s), and λ_(adv) are weights of the second lossfunction, the third loss function, and the fourth loss functionrespectively, and Lintra is the first network loss value.

If the image sample group input to the neural network and the attributecoding network module is an image pair of different objects, a secondnetwork loss value of the neural network is obtained in a sixth presetmanner based on the first loss value, the second loss value, the thirdloss value, and the fourth loss value. The expression of the sixthpreset manner is as represented by formula (7):L _(inter) =L _(c)+λ_(cr) L _(cr)+λ_(s) L _(s)+λ_(adv) L _(adv)  (7)

where λ_(cr), λ_(s), and λ_(adv) are weights of the second lossfunction, the third loss function, and the fourth loss functionrespectively, and Linter is the second network loss value.

In the embodiments of the present disclosure, the loss value of theneural network is obtained according to a sum result of the firstnetwork loss value and the second network loss value. That is, the lossvalue of the neural network is L=Lintra+Linter. In the training process,if the obtained loss value is greater than the loss threshold, feedbackadjustment is performed on the parameters of the neural network, forexample, feedback adjustment is performed on the parameters of themultiple network modules (the identity coding network module, theattribute coding network module, the generation network module, and thediscrimination network module, etc.). The training is terminated whenthe loss value of the neural network is less than the loss threshold. Inthis case, neural network optimization is completed. Alternatively, inother embodiments, the parameters of the identity coding network module,the attribute coding network module, and the classification module mayalso be adjusted according to the first loss value, the parameters ofthe generation network module are adjusted according to the second lossvalue and the third loss value, and the parameters of the discriminationnetwork modules are adjusted according to the fourth loss value. Thetraining is terminated when the loss value is less than the lossthreshold of the corresponding loss function. That is, in theembodiments of the present disclosure, feedback adjustment and trainingmay be performed separately on any one of the network modules, andunified adjustment may also be performed on some or all network modulesof the neural network through the loss value of the neural network. Aperson skilled in the art could select an appropriate manner accordingto requirements to perform this adjustment process.

In addition, in the embodiments of the present disclosure, to improvethe recognition precision of the identity features of the neuralnetwork, it is also possible to add noise to the image before inputtingthe image sample group to the identity coding network module, forexample, adding noise to an image areas of objects in the two images inthe image pair. In the embodiments of the present disclosure, the noiseis added by adding a coverage layer to a part of an image area of acharacter object, and the size of the coverage layer can be set a personskilled in the art according to requirements, and is not limited in thepresent disclosure. It should be noted herein that, in the embodimentsof the present disclosure, noise is only added to the image input to theidentity coding network module, and no noise is introduced into othernetwork modules. In this manner, the precision of identity recognitionof the neural network is effectively improved.

To describe the embodiments of the present disclosure more clearly,training processes of the first type of sample and the second type ofsample are described below by means of examples.

FIG. 6 is a schematic diagram of a process of performing networkoptimization processing by using a first type of sample according toembodiments of the present disclosure. Two images Xu and Xv of a sameobject are input to an identity coding network Eid to obtain a firstfeature, the image Xu and Xv are input to an attribute coding network Eato obtain a second feature, and the first feature is input to aclassifier C to obtain a first classification result and a first lossvalue Lc. Noise is added to the image input to the identity codingnetwork Eid, for example, a coverage map is added to a part of an areaof a character object, to block a partial area.

The second features in the image pair are exchanged, and two new imagesafter the exchange processing are obtained by using the generationnetwork module G. In this case, a second loss value Lir corresponding tothe two new images and a third loss value Ls corresponding to the firstfeature and the second feature corresponding to the two new images areobtained, and the new image is input to the discrimination networkmodule D to obtain a fourth loss value Lady. In this case, the lossvalue of the neural network is obtained by using the first loss valueLc, the second loss value Lir, the third loss value Ls, and the fourthloss value Lady. If the loss value is less than the loss threshold, thetraining is terminated. Otherwise, feedback adjustment is performed onthe parameters of at least one network module of the neural network.

FIG. 7 is a schematic diagram of a process of performing networkoptimization processing by using a second type of sample according toembodiments of the present disclosure. Two images Xu and Xw of differentobjects are input to an identity coding network Eid to obtain a firstfeature, the image Xu and Xw are input to an attribute coding network Eato obtain a second feature, and the first feature is input to aclassifier C to obtain a first classification result and a first lossvalue Lc. Noise is added to the image input to the identity codingnetwork Eid, for example, a coverage map is added to a part of an areaof a character object, to block a partial area.

The second features in the image pair are exchanged, and twointermediate images after the exchange processing are obtained by usinga generator, and first features and second features of the twointermediate images are further obtained by using the identity codingnetwork module Eid and the attribute coding network module Ea, and thenthe second features of the intermediate images are exchanged to obtain anew image. In this case, second loss values Ls corresponding to the twonew images and third loss values Lcr corresponding to first features andsecond features corresponding to the two new images are obtained, andthe intermediate images or the new images are input to thediscrimination network module D to obtain a fourth loss value Lady. Inthis case, the loss value of the neural network is obtained by using thefirst loss value, the second loss value, the third loss value, and thefourth loss value. If the loss value is less than the loss threshold,the training is terminated. Otherwise, feedback adjustment is performedon the parameters of at least one network module of the neural network.

In the embodiments of the present disclosure, a first feature (identityfeature) and a second feature other than the first feature in an inputimage are effectively extracted, and second features of two images areexchanged to form a new picture, so that identity-related features andidentity-independent features are successfully separated, where theidentity-related features may be effectively used for pedestrianre-identification. In the embodiments of the present disclosure, noauxiliary information other than an image data set is required in atraining stage and an application stage, sufficient generationsupervision is provided, and recognition precision is effectivelyimproved.

A person skilled in the art could understand that, in the foregoingmethod in the detailed description, the writing sequence of the stepsdoes not mean a strict execution sequence, and does not constitute anylimitation on the implementation process. The specific executionsequence of the steps should be determined based on functions andpossible internal logic of the steps.

In addition, the embodiments of the present disclosure further providean image processing method. The method is applied to the neural networkobtained through the image optimization method provided in the firstaspect to perform operations of image recognition, to obtain arecognition result corresponding to the identity of the input image.

FIG. 8 is a flowchart of an imaging processing method according toembodiments of the present disclosure. The method includes:

S10: receiving an input image;

S20: recognizing a first feature of the input image by means of a neuralnetwork model;

and

S30: determining an identity of an object in the input image based onthe first feature, where the neural network model is a network modelobtained after optimization processing through the network optimizationmethod according to any item in the first aspect.

In the embodiments of the present disclosure, a neural network modelthat meets requirements is obtained through training according to thefirst aspect, and an operation of recognizing an object in an image isperformed by using the neural network model. That is, an imageprocessing apparatus capable of performing operations such as imagerecognition is formed by using the neural network model, and theapparatus is configured to perform the foregoing identity recognitionprocess.

In the embodiments of the present disclosure, a database is included.The database includes information about multiple person objects, such asimages of the person objects and corresponding identity information,such as information about names, ages, and occupations. No limitation ismade thereto in the present disclosure.

After the input image is received, in the embodiments of the presentdisclosure, the first feature of the received input image is comparedwith the images of the person objects in the database to determine aperson object that matches the received input image in the database. Theneural network model in the embodiments of the present disclosure istrained in the foregoing embodiments, and meets the precisionrequirements. Therefore, in the embodiments of the present disclosure,the object that matches the input image can be precisely determined, andthen identity information corresponding to the object is obtained.

The image processing method in the embodiments of the present disclosurecan be used to quickly recognize the identity of an image object, andcan improve the recognition precision.

It can be understood that the foregoing method embodiments mentioned inthe present disclosure can be combined with each other to form combinedembodiments without departing from principle logic. Details are notdescribed in the present disclosure repeatedly due to space limitation.

In addition, the present disclosure further provides an image processingapparatus, an electronic device, a computer-readable storage medium, anda program, which are all configured to implement any image processingmethod provided in the present disclosure. For corresponding technicalsolutions and descriptions, refer to corresponding descriptions of themethod part. Details are not described again.

FIG. 9 is a block diagram of a network optimization apparatus accordingto embodiments of the present disclosure. As shown in FIG. 9, thenetwork optimization apparatus includes:

an obtaining module 10, configured to obtain an image sample group,where the image sample group includes an image pair formed by images ofa same object and an image pair formed by images of different objects; afeature coding network module 20, configured to obtain a first featureand a second feature of an image in the image sample group; aclassification module 30, configured to obtain a first classificationresult according to the first feature of the image; a generation networkmodule 40, configured to perform feature exchange processing on theimage pair in the image sample group to obtain a new image pair, wherethe feature exchange processing is to generate a new first image byusing a first feature of a first image and a second feature of a secondimage in the image pair, and to generate a new second image by using asecond feature of the first image and a first feature of the secondimage; a loss value obtaining module 50, configured to obtain a firstloss value of the first classification result, a second loss value ofthe new image pair, and a third loss value of first features and secondfeatures of the new image pair in a preset manner; and an adjustmentmodule 60, configured to adjust parameters of the neural network atleast according to the first loss value, the second loss value, and thethird loss value until a preset requirement is met.

In some possible implementations, the feature coding network moduleincludes an identity coding network module and an attribute codingnetwork module, where the obtaining module is further configured toinput two images in the image pair to the identity coding network moduleand the attribute coding network module; and the identity coding networkmodule is configured to: obtain first features of the two images in theimage pair by using the identity coding network module, and obtainsecond features of the two images in the image pair by using theattribute coding network module.

In some possible implementations, the loss value obtaining module isfurther configured to: obtain the first classification result of thefirst features obtained by means of the identity coding network module;and obtain the first loss value in a first preset manner according tothe first classification result and a real classification resultcorresponding to the image in the image sample group.

In some possible implementations, the apparatus further includes: apreprocessing module, configured to add noise to image areas of objectsin the two images in the image pair before inputting the two images inthe image pair to the identity coding network module.

In some possible implementations, the generation network module isfurther configured to: if an input image pair includes images of a sameobject, perform feature exchange processing on the images in the imagepair once, to obtain the new image pair; and generate a new first imageby using the first feature of the first image and the second feature ofthe second image in the image pair, and generate a new second image byusing the second feature of the first image and the first feature of thesecond image.

In some possible implementations, the generation network module isfurther configured to: if the input image pair are images of differentobjects, perform feature exchange processing on the images in the imagepair twice, to obtain the new image pair; and generate a firstintermediate image by using the first feature of the first image and thesecond feature of the second image in the image pair, and generate asecond intermediate image by using the second feature of the first imageand the first feature of the second image; and generate a new firstimage by using a first feature of the first intermediate image and asecond feature of the second intermediate image, and generate a newsecond image by using a second feature of the first intermediate imageand a first feature of the second intermediate image.

In some possible implementations, the loss value obtaining module isfurther configured to obtain, in a second preset manner, the second lossvalue of the new image pair obtained by means of the network generationmodule relative to an original image pair, where the original image paircorresponds to the new image pair.

In some possible implementations, the loss value obtaining module isfurther configured to obtain the third loss value of the first featuresand the second features of the new image pair in a third preset mannerbased on the first features and the second features of the new imagepair as well as first features and second features of the correspondingoriginal image pair, where the original image pair corresponds to thenew image pair.

In some possible implementations, the apparatus further includes: adiscrimination network module, configured to: receive the new imagepair, and obtain a label feature representing reality of the new imagepair; and the loss value obtaining module is further configured toobtain a fourth loss value of the new image pair in a fourth presetmanner based on the label feature.

In some possible implementations, the adjustment module is furtherconfigured to: obtain the loss value of the neural network by using thefirst loss value, the second loss value, the third loss value, and thefourth loss value; and adjust the parameters of the neural network byusing the loss value of the neural network until the preset requirementis met.

In some possible implementations, the adjustment module is furtherconfigured to: if the image sample group input to the neural network isthe image pair of the same object, obtain a first network loss value ofthe neural network in a fifth preset manner based on the first lossvalue, the second loss value, the third loss value, and the fourth lossvalue; if the image sample group input to the neural network is theimage pair of different objects, obtain a second network loss value ofthe neural network in a sixth preset manner based on the first lossvalue, the second loss value, the third loss value, and the fourth lossvalue; and obtain the loss value of the neural network based on a sumresult of the first network loss value and the second network lossvalue.

FIG. 10 is a block diagram of an image processing apparatus according toembodiments of the present disclosure. As shown in the figure, the imageprocessing apparatus includes:

a receiving module 100, configured to receive an input image;

a recognition module 200, configured to recognize a first feature of theinput image by means of a neural network model; and

an identity determination module 300, configured to determine anidentity of an object in the input image based on the first feature.

The neural network model is a network model obtained after optimizationprocessing through the network optimization method according to any oneof the first aspect.

In some embodiments, the functions of or modules included in theapparatus provided in the embodiments of the present disclosure areconfigured to perform the methods described in the foregoing methodembodiments. For specific implementations of the methods, refer to thedescriptions of the foregoing method embodiments. For brevity, detailsare not described herein again.

The embodiments of the present disclosure further provide acomputer-readable storage medium, having computer program instructionsstored thereon, where the computer program instructions are executed bya processor to implement the foregoing methods. The computer-readablestorage medium may be a non-volatile computer-readable storage medium.

The embodiments of the present disclosure further provide an electronicdevice, including: a processor; and a memory configured to storeprocessor-executable instructions; where the processor is configured toimplement the foregoing methods.

The embodiments of the present disclosure provide a computer programproduct, including a computer-readable code, where when thecomputer-readable code runs on a device, a processor in the deviceexecutes instructions for implementing the method provided in any one ofthe foregoing embodiments.

The electronic device may be provided as a terminal, a server, or otherforms of devices.

FIG. 11 is a block diagram of an electronic device 800 according toembodiments of the present disclosure. For example, the electronicdevice 800 may be a terminal such as a mobile phone, a computer, adigital broadcast terminal, a message transceiver device, a gameconsole, a tablet device, a medical device, exercise equipment, and apersonal digital assistant.

Referring to FIG. 11, the electronic device 800 may include one or moreof the following components: a processing component 802, a memory 804, apower supply component 806, a multimedia component 808, an audiocomponent 810, an Input/Output (I/O) interface 812, a sensor component814, and a communication component 816.

The processing component 802 generally controls overall operation of theelectronic device 800, such as operations associated with display, phonecalls, data communications, camera operations, and recording operations.The processing component 802 may include one or more processors 820 toexecute instructions to implement all or some of the steps of the methodabove. In addition, the processing component 802 may include one or moremodules to facilitate interaction between the processing component 802and other components. For example, the processing component 802 mayinclude a multimedia module to facilitate interaction between themultimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to supportoperations on the electronic device 800. Examples of the data includeinstructions for any application or method operated on the electronicdevice 800, contact data, contact list data, messages, pictures, videos,and etc. The memory 804 may be implemented by any type of volatile ornon-volatile storage device, or a combination thereof, such as a StaticRandom-Access Memory (SRAM), an Electrically Erasable ProgrammableRead-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory(EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory(ROM), a magnetic memory, a flash memory, a disk or an optical disk.

The power supply component 806 provides power for various components ofthe electronic device 800. The power supply component 806 may include apower management system, one or more power supplies, and othercomponents associated with power generation, management, anddistribution for the electronic device 800.

The multimedia component 808 includes a screen between the electronicdevice 800 and a user that provides an output interface. In someembodiments, the screen may include a Liquid Crystal Display (LCD) and aTouch Panel (TP). If the screen includes a TP, the screen may beimplemented as a touch screen to receive input signals from the user.The TP includes one or more touch sensors for sensing touches, swipes,and gestures on the TP. The touch sensor may not only sense the boundaryof a touch or swipe action, but also detect the duration and pressurerelated to the touch or swipe operation. In some embodiments, themultimedia component 808 includes a front-facing camera and/or arear-facing camera. When the electronic device 800 is in an operationmode, for example, a photography mode or a video mode, the front-facingcamera and/or the rear-facing camera may receive external multimediadata. Each of the front-facing camera and the rear-facing camera may bea fixed optical lens system, or have focal length and optical zoomcapabilities.

The audio component 810 is configured to output and/or input an audiosignal. For example, the audio component 810 includes a microphone(MIC), and the microphone is configured to receive an external audiosignal when the electronic device 800 is in an operation mode, such as acalling mode, a recording mode, and a voice recognition mode. Thereceived audio signal may be further stored in the memory 804 ortransmitted by means of the communication component 816. In someembodiments, the audio component 810 further includes a speaker foroutputting the audio signal.

The I/O interface 812 provides an interface between the processingcomponent 802 and a peripheral interface module, which may be akeyboard, a click wheel, a button, etc. The button may include, but isnot limited to, a home button, a volume button, a start button, and alock button.

The sensor component 814 includes one or more sensors for providingstate assessment in various aspects for the electronic device 800. Forexample, the sensor component 814 may detect an on/off state of theelectronic device 800, and relative positioning of components, which arethe display and keypad of the electronic device 800, for example, andthe sensor component 814 may further detect a position change of theelectronic device 800 or a component of the electronic device 800, thepresence or absence of contact of the user with the electronic device800, the orientation or acceleration/deceleration of the electronicdevice 800, and a temperature change of the electronic device 800. Thesensor component 814 may include a proximity sensor, which is configuredto detect the presence of a nearby object when there is no physicalcontact. The sensor component 814 may further include a light sensor,such as a CMOS or CCD image sensor, for use in an imaging application.In some embodiments, the sensor component 814 may further include anacceleration sensor, a gyroscope sensor, a magnetic sensor, a pressuresensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired orwireless communications between the electronic device 800 and otherdevices. The electronic device 800 may access a wireless network basedon a communication standard, such as WiFi, 2G, or 3G, or a combinationthereof. In an exemplary embodiment, the communication component 816receives a broadcast signal or broadcast-related information from anexternal broadcast management system by means of a broadcast channel. Inan exemplary embodiment, the communication component 816 furtherincludes a Near Field Communication (NFC) module to facilitateshort-range communication. For example, the NFC module may beimplemented based on Radio Frequency Identification (RFID) technology,Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB)technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implementedby one or more Application-Specific Integrated Circuits (ASICs), DigitalSignal Processors (DSPs), Digital Signal Processing Devices (DSPDs),Programmable Logic Devices (PLDs), Field-Programmable Gate Arrays(FPGAs), controllers, microcontrollers, microprocessors, or otherelectronic elements, to execute the method above.

In an exemplary embodiment, a non-volatile computer-readable storagemedium is further provided, for example, a memory 804 including computerprogram instructions, which can executed by the processor 820 of theelectronic device 800 to implement the methods above.

FIG. 12 is a block diagram of an electronic device 1900 according to anexemplary embodiment. For example, the electronic device 1900 may beprovided as a server. Referring to FIG. 12, the electronic device 1900includes a processing component 1922 which further includes one or moreprocessors, and a memory resource represented by a memory 1932 andconfigured to store instructions executable by the processing component1922, for example, an application program. The application programstored in the memory 1932 may include one or more modules, each of whichcorresponds to a set of instructions. Further, the processing component1922 may be configured to execute instructions so as to execute theabove method.

The electronic device 1900 may further include a power supply component1926 configured to execute power management of the electronic device1900, a wired or wireless network interface 1950 configured to connectthe electronic device 1900 to the network, and an I/O interface 1958.The electronic device 1900 may be operated based on an operating systemstored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™,Linux™, FreeBSD™ or the like.

In an exemplary embodiment, a non-volatile computer-readable storagemedium is further provided, for example, a memory 1932 includingcomputer program instructions, which can executed by the processingcomponent 1922 of the electronic device 1900 to implement the methodabove.

The present disclosure may be a system, a method, and/or a computerprogram product. The computer program product may include acomputer-readable storage medium, on which computer-readable programinstructions used by the processor to implement various aspects of thepresent disclosure are stored.

The computer-readable storage medium may be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer-readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. More specific examples (a non-exhaustive list) of thecomputer-readable storage medium include: a portable computer diskette,a hard disk, a Random Access Memory (RAM), an ROM, an EPROM (or a flashmemory), a SRAM, a portable Compact Disk Read-Only Memory (CD-ROM), aDigital Versatile Disc (DVD), a memory stick, a floppy disk, amechanically encoded device such as punch-cards or raised structure in agroove having instructions stored thereon, and any suitable combinationthereof. A computer-readable storage medium, as used herein, is not tobe construed as being transitory signals per se, such as radio waves orother freely propagating electromagnetic waves, electromagnetic wavespropagating through a waveguide or other transmission media (e.g., lightpulses passing through a fiber-optic cable), or electrical signalstransmitted through a wire.

Computer-readable program instructions described herein can bedownloaded to respective computing/processing devices from acomputer-readable storage medium or to an external computer or externalstorage device via a network, for example, the Internet, a Local AreaNetwork (LAN), a wide area network and/or a wireless network. Thenetwork may include copper transmission cables, optical transmissionfibers, wireless transmission, routers, firewalls, switches, gatewaycomputers and/or edge servers. A network adapter card or networkinterface in each computing/processing device receives computer-readableprogram instructions from the network and forwards the computer-readableprogram instructions for storage in a computer-readable storage mediumwithin the respective computing/processing device.

Computer program instructions for carrying out operations of the presentdisclosure may be assembler instructions, Instruction-Set-Architecture(ISA) instructions, machine instructions, machine dependentinstructions, microcode, firmware instructions, state-setting data, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++ or the like, and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The computer-readable program instructions mayexecute entirely on the user's computer, partly on the user's computer,as a stand-alone software package, partly on the user's computer andpartly on a remote computer or entirely on the remote computer orserver. In a scenario involving a remote computer, the remote computermay be connected to the user's computer through any type of network,including a LAN or a Wide Area Network (WAN), or the connection may bemade to an external computer (for example, through the Internet using anInternet service provider). In some embodiments, an electronic circuitsuch as a programmable logic circuit, an FPGA, or a Programmable LogicArray (PLA) is personalized by using status information of the computerreadable program instructions, and the electronic circuit may executethe computer readable program instructions to implement various aspectsof the present disclosure.

Various aspects of the present disclosure are described here withreference to the flowcharts and/or block diagrams of the methods,apparatuses (systems), and computer program products according to theembodiments of the present disclosure. It should be understood that eachblock of the flowcharts and/or block diagrams, and combinations of theblocks in the flowcharts and/or block diagrams can be implemented bycomputer-readable program instructions.

These computer-readable program instructions may be provided to aprocessor of a general-purpose computer, special-purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer-readable program instructionsmay also be stored in a computer-readable storage medium that can causea computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that thecomputer-readable medium having instructions stored therein includes anarticle of manufacture instructing instructions which implement theaspects of the functions/acts specified in one or more blocks of theflowcharts and/or block diagrams.

The computer-readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus or other device implement thefunctions/acts specified in one or more blocks of the flowcharts and/orblock diagrams.

The flowcharts and block diagrams in the accompanying drawings showarchitectures, functions, and operations that may be implemented by thesystems, methods, and computer program products in the embodiments ofthe present disclosure. In this regard, each block in the flowchart ofblock diagrams may represent a module, segment, or portion ofinstruction, which includes one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may also occur out ofthe order noted in the accompanying drawings. For example, two blocksshown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It should also benoted that each block of the block diagrams and/or flowcharts, andcombinations of blocks in the block diagrams and/or flowcharts, can beimplemented by special purpose hardware-based systems that perform thespecified functions or acts or carried out by combinations of specialpurpose hardware and computer instructions.

The embodiments of the present disclosure are described above. Theforegoing descriptions are exemplary but not exhaustive, and are notlimited to the embodiments of the disclosure. Many modifications andvariations will be apparent to persons of ordinary skill in the artwithout departing from the scope and spirit of the describedembodiments. The terms used herein are intended to best explain theprinciples of the embodiments, practical applications, or technicalimprovements to the technologies in the market, or to enable otherpersons of ordinary skill in the art to understand the embodimentsdisclosed herein.

The invention claimed is:
 1. A network optimization method foroptimizing a neural network, comprising: obtaining an image samplegroup, wherein the image sample group comprises an image pair formed byimages of a same object and an image pair formed by images of differentobjects; obtaining a first feature and a second feature of an image inthe image sample group, and obtaining a first classification result byusing the first feature of the image, wherein the first featurecomprises an identity feature, and the second feature comprises anattribute feature; performing feature exchange processing on an imagepair in the image sample group to obtain a new image pair, wherein thefeature exchange processing is to generate a new first image by using afirst feature of a first image and a second feature of a second image inthe image pair, and to generate a new second image by using a secondfeature of the first image and a first feature of the second image;obtaining a first loss value of the first classification result, asecond loss value of the new image pair, and a third loss value of firstfeatures and second features of the new image pair in a preset manner;and adjusting parameters of the neural network at least according to thefirst loss value, the second loss value, and the third loss value untila preset requirement is met.
 2. The method according to claim 1, whereinobtaining the first feature and the second feature of the image in theimage sample group comprises: inputting two images in the image pair toan identity coding network module and an attribute coding network moduleof the neural network; and obtaining first features of the two images inthe image pair by using the identity coding network module, andobtaining second features of the two images in the image pair by usingthe attribute coding network module.
 3. The method according to claim 2,wherein obtaining the first loss value of the first classificationresult, the second loss value of the new image pair, and the third lossvalue of the first features and the second features of the new imagepair in the preset manner comprises: obtaining the first classificationresult of the first features obtained by means of the identity codingnetwork module; and obtaining the first loss value in a first presetmanner according to the first classification result and a realclassification result corresponding to the image in the image samplegroup.
 4. The method according to claim 2, wherein before inputting thetwo images in the image pair to the identity coding network module, themethod further comprises: adding noise to image areas of objects in thetwo images in the image pair.
 5. The method according to claim 1,wherein performing feature exchange processing on the image pair in theimage sample group to obtain the new image pair comprises: inputting afirst feature and a second feature of an image in the image pair in theimage sample group to a generation network module of the neural network;and performing the feature exchange processing on the image pair in theimage sample group by means of the generation network module to obtainthe new image pair.
 6. The method according to claim 1, wherein if aninput image pair comprises images of a same object, performing featureexchange processing on the image pair in the image sample group toobtain the new image pair comprises: performing the feature exchangeprocessing on the images in the image pair once to obtain the new imagepair, and performing the feature exchange processing on the images inthe image pair once to obtain the new image pair comprises: generating anew first image by using the first feature of the first image and thesecond feature of the second image in the image pair, and generating anew second image by using the second feature of the first image and thefirst feature of the second image, and/or wherein if the input imagepair are images of different objects, performing feature exchangeprocessing on the image pair in the image sample group to obtain the newimage pair comprises: performing the feature exchange processing on theimages in the image pair twice to obtain the new image pair, andperforming the feature exchange processing on the images in the imagepair twice to obtain the new image pair comprises: generating a firstintermediate image by using the first feature of the first image and thesecond feature of the second image in the image pair, and generating asecond intermediate image by using the second feature of the first imageand the first feature of the second image; and generating a new firstimage by using a first feature of the first intermediate image and asecond feature of the second intermediate image, and generating a newsecond image by using a second feature of the first intermediate imageand a first feature of the second intermediate image.
 7. The methodaccording to claim 5, wherein obtaining the first loss value of thefirst classification result, the second loss value of the new imagepair, and the third loss value of the first features and the secondfeatures of the new image pair in the preset manner comprises:obtaining, in a second preset manner, the second loss value of the newimage pair obtained by means of the network generation module relativeto an original image pair, wherein the original image pair correspondsto the new image pair.
 8. The method according to claim 1, whereinobtaining the first loss value of the first classification result, thesecond loss value of the new image pair, and the third loss value of thefirst features and the second features of the new image pair in thepreset manner comprises: obtaining the third loss value of the firstfeatures and the second features of the new image pair in a third presetmanner based on the first features and the second features of the newimage pair as well as first features and second features of the originalimage pair, wherein the original image pair corresponds to the new imagepair.
 9. The method according to claim 1, wherein after performingfeature exchange processing on the image pair in the image sample groupto obtain the new image pair, the method further comprises: inputtingthe generated new image pair to a discrimination network module of theneural network to obtain a label feature representing reality of the newimage pair; and obtaining a fourth loss value of the new image pair in afourth preset manner based on the label feature.
 10. The methodaccording to claim 9, wherein adjusting the parameters of the neuralnetwork at least according to the first loss value, the second lossvalue, and the third loss value until the preset requirement is metcomprises: obtaining a loss value of the neural network by using thefirst loss value, the second loss value, the third loss value, and thefourth loss value; and adjusting the parameters of the neural network byusing the loss value of the neural network until the preset requirementis met.
 11. The method according to claim 10, wherein obtaining the lossvalue of the neural network by using the first loss value, the secondloss value, the third loss value, and the fourth loss value comprises:if the image sample group input to the neural network is the image pairof the same object, obtaining a first network loss value of the neuralnetwork in a fifth preset manner based on the first loss value, thesecond loss value, the third loss value, and the fourth loss value; ifthe image sample group input to the neural network is the image pair ofdifferent objects, obtaining a second network loss value of the neuralnetwork in a sixth preset manner based on the first loss value, thesecond loss value, the third loss value, and the fourth loss value; andobtaining the loss value of the neural network based on a sum result ofthe first network loss value and the second network loss value.
 12. Animage processing apparatus, comprising: a processor; and a memoryconfigured to store processor-executable instructions, wherein theprocessor is configured to invoke the instructions stored in the memory,so as to: obtain an image sample group, wherein the image sample groupcomprises an image pair formed by images of a same object and an imagepair formed by images of different objects; obtain a first feature and asecond feature of an image in the image sample group; obtain a firstclassification result according to the first feature of the image;perform feature exchange processing on an image pair in the image samplegroup to obtain a new image pair, wherein the feature exchangeprocessing is to generate a new first image by using a first feature ofa first image and a second feature of a second image in the image pair,and to generate a new second image by using a second feature of thefirst image and a first feature of the second image; obtain a first lossvalue of the first classification result, a second loss value of the newimage pair, and a third loss value of first features and second featuresof the new image pair in a preset manner; and adjust parameters of theneural network at least according to the first loss value, the secondloss value, and the third loss value until a preset requirement is met.13. The apparatus according to claim 12, wherein obtaining the firstfeature and the second feature of the image in the image sample groupcomprises: inputting two images in the image pair to an identity codingnetwork module and an attribute coding network module of the neuralnetwork, and obtaining first features of the two images in the imagepair by using the identity coding network module, and obtaining secondfeatures of the two images in the image pair by using the attributecoding network module.
 14. The apparatus according to claim 13, whereinobtaining the first loss value of the first classification result, thesecond loss value of the new image pair, and the third loss value of thefirst features and the second features of the new image pair in thepreset manner comprises: obtaining the first classification result ofthe first features obtained by means of the identity coding networkmodule; and obtaining the first loss value in a first preset manneraccording to the first classification result and a real classificationresult corresponding to the image in the image sample group.
 15. Theapparatus according to claim 13, wherein the processor is furtherconfigured to: add noise to image areas of objects in the two images inthe image pair before inputting the two images in the image pair to theidentity coding network module.
 16. The apparatus according to claim 12,wherein if an input image pair comprises images of a same object,performing feature exchange processing on the image pair in the imagesample group to obtain the new image pair comprises: performing thefeature exchange processing on the images in the image pair once toobtain the new image pair, and performing the feature exchangeprocessing on the images in the image pair once to obtain the new imagepair comprises: generating a new first image by using the first featureof the first image and the second feature of the second image in theimage pair, and generating a new second image by using the secondfeature of the first image and the first feature of the second image,and/or wherein if the input image pair are images of different object,performing feature exchange processing on the image pair in the imagesample group to obtain the new image pair comprises: performing thefeature exchange processing on the images in the image pair twice toobtain the new image pair, and performing the feature exchangeprocessing on the images in the image pair twice to obtain the new imagepair comprises: generating a first intermediate image by using the firstfeature of the first image and the second feature of the second image inthe image pair, and generating a second intermediate image by using thesecond feature of the first image and the first feature of the secondimage; and generating a new first image by using a first feature of thefirst intermediate image and a second feature of the second intermediateimage, and generating a new second image by using a second feature ofthe first intermediate image and a first feature of the secondintermediate image.
 17. The apparatus according to claim 16, whereinobtaining the first loss value of the first classification result, thesecond loss value of the new image pair, and the third loss value of thefirst features and the second features of the new image pair in thepreset manner comprises: obtaining, in a second preset manner, thesecond loss value of the new image pair obtained by means of the networkgeneration module relative to an original image pair, wherein theoriginal image pair corresponds to the new image pair.
 18. The apparatusaccording to claim 12, wherein obtaining the first loss value of thefirst classification result, the second loss value of the new imagepair, and the third loss value of the first features and the secondfeatures of the new image pair in the preset manner comprises: obtainingthe third loss value of the first features and the second features ofthe new image pair in a third preset manner based on the first featuresand the second features of the new image pair as well as first featuresand second features of the original image pair, wherein the originalimage pair corresponds to the new image pair.
 19. The apparatusaccording to claim 12, wherein the processor is further configured to:receive the new image pair, and obtain a label feature representingreality of the new image pair; and obtain a fourth loss value of the newimage pair in a fourth preset manner based on the label feature.
 20. Anon-transitory computer-readable storage medium, having computer programinstructions stored thereon, wherein when the computer programinstructions are executed by a processor, the processor is caused toperform the operations of: obtaining an image sample group, wherein theimage sample group comprises an image pair formed by images of a sameobject and an image pair formed by images of different objects;obtaining a first feature and a second feature of an image in the imagesample group, and obtaining a first classification result by using thefirst feature of the image, wherein the first feature comprises anidentity feature, and the second feature comprises an attribute feature;performing feature exchange processing on an image pair in the imagesample group to obtain a new image pair, wherein the feature exchangeprocessing is to generate a new first image by using a first feature ofa first image and a second feature of a second image in the image pair,and to generate a new second image by using a second feature of thefirst image and a first feature of the second image; obtaining a firstloss value of the first classification result, a second loss value ofthe new image pair, and a third loss value of first features and secondfeatures of the new image pair in a preset manner; and adjustingparameters of the neural network at least according to the first lossvalue, the second loss value, and the third loss value until a presetrequirement is met.